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The number of online courses, programs, and 
schools are growing exponentially in K-12 education. Given 
the unique nature of online courses and the distinct skills nec¬ 
essary to create a quality online course, it is essential that ef¬ 
fective professional development be provided for teachers de¬ 
signing online courses. Online courses need to be of the same 
quality as their face-to-face counterparts. Thus, K-12 teachers 
must learn how to design an online course to meet recognized 
standards of quality. The current study explored the effects of 
a professional development model designed to provide an au¬ 
thentic, project- and standards-based experience to K-12 edu¬ 
cators. The extent to which teachers designed an online mod¬ 
ule to meet Quality Matters standards was assessed. Overall, 
the majority of participants (n=97) were able to successfully 
transfer learning to design and develop an online course to 
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meet standards. Areas of concern regarding specific standards 
are discussed. Recommendations for more rigorous research 
are suggested to determine the degree to which outcomes 
are specifically related to the course experience as opposed 
to outside variables such as learner and instructor character¬ 
istics. Results of this study can be used to inform design of 
effective professional development for online course design. 


INTRODUCTION 

Over the past decade, the landscape of K-12 education has seen dramatic 
change in the way instruction is delivered, namely utilization of the Internet 
to supplement or even replace the face-to-face classroom. Online courses fo¬ 
cused on K-12 learners are becoming more prevalent nationwide, meeting 
evolving needs for a diverse population of students including: centralizing 
expert teachers for advanced placement and post-baccalaureate programs, of¬ 
fering new options as opposed to alternative schools, and providing consis¬ 
tency in educational practices for homebound and virtual students (Powell & 
Patrick, 2006). Given this growing trend, and the complicated nature of de¬ 
signing online courses (Wray, Lowenthal, Bates, & Stevens, 2008), it is cru¬ 
cial to ensure that K-12 teachers are provided with the necessary design skills 
to deliver quality online courses (Chiu, 2013). 

Professional development is fundamental to changing the way teachers 
teach, regardless of the learning environment (Rice & Dawley, 2009). A ma¬ 
jority of teachers design and develop the courses they deliver (Powell, 2010). 
Therefore, it is vital to provide effective professional development for teach¬ 
ers to ascertain the necessary skills to create quality courses. The question is, 
which approaches to professional development for these skills are most ef¬ 
fective? Unfortunately, the lack of empirical research specific to professional 
development for online course design leaves us with few, if any, real answers 
to this question. 

According to Mercer (2014), participating in a two-day, face-to-face pro¬ 
fessional development workshop focused on standards-based design can in¬ 
crease knowledge of best practice in online course design, but it is not suf¬ 
ficient to aide teachers in how to apply standards of quality to design online 
courses. Moreover, when teachers are left to apply standards to online cours¬ 
es without the assistance of experts in instructional design, the outcome is a 
course that still lacks alignment among assessments and learning objectives 
(Mercer, 2014). 

Mercer (2014) proposed a standards-based, online model for professional 
development. The model guides participants through six weeks of funda- 
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mental design principles while they simultaneously build online instruction 
to meet nationally-recognized standards of quality consistent with the Quali¬ 
ty Matters program. Critical to the success of the model is the continuous for¬ 
mative feedback provided to participants as they design an online course to 
meet QM standards. The present study evaluated the proposed model known 
as the Virtual Instructor Certificate Program by measuring outcomes of K-12 
participants using the essential standards of the Quality Matters Rubric as the 
benchmark for success. 


BACKGROUND 

There is little research on effective professional development for K-12 
educators learning to design an online course, much less, a course that 
meets national standards of quality (Shattuck, 2013). Shattuck points out 
that research continues to focus on comparison studies (Means, Toyama, 
Bakia, & Jones, 2010), how to meet the needs of learners with distinguish¬ 
ing characteristics such as at-risk students (Archambault et al., 2010), stu¬ 
dents with special needs (Thompson, Ferdig, & Black, 2012), and those 
who are considered non-traditional students (Sturgis, Rath, Weisstein, & 
Patrick, 2010). 

Cavanaugh, Barbour, and Clark (2009) conducted a review of research 
and practice in K-12 online learning, revealing not only the limited pub¬ 
lished research on K-12 distance education but also the limited focus 
on specific research related to instructional design as it pertains to online 
courses developed for K-12 students. Shattuck (2013) expounded upon this 
review discovering that the literature available specifically related to in¬ 
structional design and online learning in the K-12 distance education arena 
is “virtually non-existent” (Shattuck, 2013, p. 3), indicating a clear and con¬ 
tinued gap in the literature on a most important topic given the enormous 
growth of online learning in K-12 schools. However, a more recent review 
of literature has shown promise in published research related to K-12 dis¬ 
tance education with a marked increase in attention to teacher training 
(Shattuck, 2015). What follows is a discussion of some of the most salient 
points to be made regarding professional development for K-12 educators 
learning to transfer skills for lesson design to the online classroom. 

Tenets of Effective K-12 Professional Development in Online Course Design 

Authenticity. Authentic project-based learning is an essential element 
necessary for an effectively designed online course (Dabner, Davis, & Zaka, 
2012). Authentic project-based learning applies to both the courses offered 
to students as well as those courses that teach instructors how to design a 
course online. Authentic professional development provides an opportunity 
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for instructors to apply newly acquired skills to situations that mirror what 
their “real” teaching experience will entail. Guidelines released by The Na¬ 
tional Education Association, The Southern Regional Education Board, and 
The International Association for K-12 Online Learning recommend that 
teachers should experience learning in the same way as their students (iNA- 
COL, 2011; NEA, 2006; SREC, 2009). Therefore, effective professional de¬ 
velopment for online course design would be most appropriately delivered 
via online instruction. 

Relevance 

To promote learner engagement, instructional material intended for the 
adult learner should also be relevant to participant needs (Keller, 1984). Au¬ 
thenticity during instruction creates a natural relevance to future skill use 
and is thus also a central component to effective course design. Researchers 
agree that the virtue of project-based learning supports the need for differ¬ 
entiated professional development sessions (Dabner, Davis, & Zaka, 2012; 
Duncan & Barnett, 2009; DiPietro, Ferdig, Black & Preston, 2008; Jung 
2005). Agyei and Voogt (2014) investigated the extent to which beginning 
teachers transferred their learning after participating in a professional devel¬ 
opment program designed to help teachers integrate Information Communi¬ 
cation Technology in their mathematics classes. Unfortunately, the research¬ 
ers found that transfer was negatively impacted by a lack of infrastructure to 
support the initiative, an issue that has arisen in past studies as well (Marek, 
2009). 

Rice and Dawley (2009) highlighted factors that would impact the deliv¬ 
ery and design of professional development opportunities for K-12 stake¬ 
holders. Of those factors, context played an integral role in how profession¬ 
al development was delivered. Given the differing influences on contexts 
and a subsequent need for a differentiated approach in professional devel¬ 
opment, specific guidelines to standardize the information communicated 
through professional development sessions are crucial. 

Standards based 

Effective professional development should incorporate learning objec¬ 
tives that are standards-aligned (Yamaguchi & Gallagher, 2007). Depend¬ 
ing on the context, standards for online course design can vary. The Inter¬ 
national Association for K-12 Online Learning (iNACOL) has developed 
a set of standards to be used for designing and delivering online courses. 
Wayer (2013) investigated the extent to which five K-12 educators applied 
these standards to blended courses after participation in an online profes¬ 
sional development course. The course focused on the iNACOL stan¬ 
dards, utilized project-based learning, as well as a community of inquiry. 
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According to Wayer, the most problematic outcome was the lack of course 
instruction provided to engage students in active learning. However, over¬ 
all, the participants were able to transfer concepts from the online profes¬ 
sional development course to each respective blended course to be taught in 
the upcoming semester. 

Not only is it important to teach educators to apply standards to the de¬ 
sign of online courses, but the professional development itself should be 
aligned with standards to improve outcomes. Holmes, Signer, and MacLeod 
(2011) used the New York State Learning Standards and National Educa¬ 
tional Technology Standards for Teachers, to design a five-week online pro¬ 
fessional development course for in-service K-12 teachers and subsequently 
measure outcomes in terms of satisfaction, course quality, and overall per¬ 
spectives of online professional development. Results suggested that the 
best method for teaching how to design an online course may be through 
online professional development, providing an authentic context where the 
teacher has the opportunity to experience the online learner perspective to 
inform future design. 

However, the existence of standards does not imply use. Although guide¬ 
lines and standards exist, the administrators fail to adhere to them (Rice 
& Dawley, 2009). In fact, similar to higher education, many online high 
schools do not require teachers to participate in online education prior to fa¬ 
cilitating online courses. The lack of required training could be one reason 
available research on effective training is limited. If standards exist, why are 
teachers not required to follow them? Hathaway and Norton (2012) posited 
that the demand for K-12 online courses exceeds the supply of available 
teachers who are adequately trained in online course delivery. This, coupled 
with stringent budgets, could be the reason school districts sometimes forgo 
systematized, standardized, online professional development, and opt in¬ 
stead for one or two days of intensive face-to-face workshops (Hathaway & 
Norton, 2012). 

Quality matters 

In addition to iNACOL and local standards, some K-12 institutions are 
using the Quality Matters™ (QM) program either in part or comprehen¬ 
sively to implement methods of quality control. The program is recognized 
world-wide as a highly reputable method for quality assurance in online 
learning (Ralston-Berg & Nath, 2008) and uses a rigorous peer-review pro¬ 
cess to assess the design of an online course (Legon & Runyon, 2007). In 
fact, the program offers a rubric specific to online course design for K-12 
distance educators to both guide design of courses and assess the quality of 
existing courses. The program’s website offers access to its rubrics, but full 
disclosure of annotations for specific review standards is only available to 
subscribing institutions. 
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Because of QM’s growing popularity, researchers are beginning to explore 
the effectiveness of the program’s professional development opportunities to 
aide application of standards to the design of online courses. However, avail¬ 
able research seems to focus on higher education, representing a lack of fo¬ 
cus in K-12 distance education research on effective professional develop¬ 
ment. Three studies investigated outcomes of participation in the program’s 
foundational workshop Applying the Quality Matters Rubric. Wright (2011) 
found that participation in the workshop increased faculty self-efficacy with 
regards to creating a quality online course. Mercer (2014) and Budzick 
(2014) further investigated variables such as knowledge of best practice in 
online course design and self-perception of the quality of the individual’s on¬ 
line course. Mercer found that while participation in the workshop can signif¬ 
icantly increase knowledge of best practice statistically, more was needed for 
successful application of standards. In addition, Budzick found that participa¬ 
tion increased willingness to use the rubric to facilitate course design. These 
studies help us to have a baseline for effective professional development, but 
more is needed. 

Regardless of why, there continues to be a lack in the existing literature 
to address impact of standards-based online professional development espe¬ 
cially related to K-12 course design. Thus, it is beneficial to use available 
research from higher education studies to guide K-12 professional develop¬ 
ment endeavors.. 

Professional Development Outcomes in Higher Education 

Shattuck (2013) noted that much of the available literature in distance 
education that is conducted in higher education has been applied to K-12 as 
well. Similarly, we can take what little research exists on professional devel¬ 
opment for online course design focused in the higher education setting and 
utilize these outcomes in hopes to build a beginning foundation of how best 
to proceed in K-12. It is important to keep in mind that even studies focused 
on faculty in higher education are limited in terms of providing empirical 
evidence on the effects of specific professional development programs as de¬ 
scribed in the previous section on QM-focused research in this area. Howev¬ 
er, available research tends to be primarily descriptive with quantitative mea¬ 
sures emphasizing self-report as opposed to objective measures of learning 
gains (Mercer, 2014). 

Faculty new to online learning can feel inadequate based on limited 
technology expertise (Berge & Muilenburg, 2000; Covington, Petherbridge, 
& Warren, 2005), concerned that converting a course to an online format 
will inevitably reduce the quality of learning (Bower, 2001). As mentioned 
earlier, putting a course online takes a distinctly different set of skills (Wol¬ 
cott & Shattuck, 2007). Therefore, increasing online self-efficacy could be 
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considered an important goal of professional development for online faculty 
and an important contributor to quality assurance. Wright (2011) investigated 
faculty perceptions regarding their ability to design, develop, and deliver an 
online course. He found a statistically significant difference for online self- 
efficacy after faculty completed a Quality Matters™ workshop. 

Powell (2010) reported that faculty who completed a total of 11 modules, 
including seven, two-hour face-to-face sessions also felt more confident. In 
addition, faculty described the professional development as useful for design 
and delivery (31%), and effective for selecting appropriate software (30%). 
However, only 27% of faculty felt confident to teach online, suggesting that 
increasing confidence is not enough, and perhaps the effectiveness of profes¬ 
sional development should be measured in multiple ways. 

Reilly, Vandenhouten, Gallagher-Lepak, and Ralston-Berg (2012) inte¬ 
grated Khan’s Flexible Framework for Elearning and Communities of Prac¬ 
tice (COP) to investigate various faculty attributes relative to elearning and 
technology. This multi-institutional approach to COP included video con¬ 
ferencing, campus leadership, annual face-to-face conferences, and online 
courses over a period of five years. Using self-report surveys, faculty de¬ 
scribed a) an increase in overall knowledge and understanding of elearning, 
b) an increase in ability to evaluate design and delivery methods for online 
learning, and c) an intent to redesign current courses based on knowledge 
gained. 

Necessary Competencies for Designing Quality Online Courses 

Although there is not extensive research on the effectiveness of specific 
professional development models for teaching teachers how to design quality 
online courses (Mercer, 2014), much less specific to K-12 (DiPietro, Ferdig, 
Black, & Preston, 2008), we do have recommendations for instructional de¬ 
sign competencies that should be part of the online K-12 educator’s knowl¬ 
edge base. One of the most notable set of guidelines for online teaching was 
provided by Goodyear, Salmon, Spector, Steeples, and Tickner (2001). More 
recently, Rozitis (2014) used a Delphi study using experts from a range of 
organizations including the Association for Educational Communications and 
Technology (AECT) to provide at the minimum, expectations for what those 
who are designing courses should know or be able to do. Outcomes specif¬ 
ic to design included: presenting accurate information, appropriately using 
technologies to enhance learning, effectively arranging media and content 
to maximize learning, creating and modifying engaging content and assess¬ 
ments, employing effective navigation and structure strategies, appropriately 
selecting technological resources, and designing strategies to encourage ac¬ 
tive learning and collaboration. 
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The lack of research available to guide districts, school administrators, 
and teachers on how to best prepare for delivering online courses to K-12 
students necessitates that distance education researchers put more focus on 
professional development geared towards online course design. Traditional 
best practices reported in the literature typically center on course delivery. 
Some of these best practices include frequently contacting with students, 
providing prompt feedback, summarizing content of discussions, monitoring 
progress, and helping students trouble shoot technical problems (Chickering 
& Gamson, 2010; Taylor & McQuiggan, 2008). Mastery of these activities 
increases faculty confidence (Wright, 2011), satisfaction, and sense of course 
ownership (Ellis & Phelps, 2000; Orr, Williams, & Pennington, 2009). Al¬ 
though these practices are essential to student success, it is the course design 
that drives how the course unfolds during the process of delivery. Recent¬ 
ly, these best practices in online course development also encourage a sys¬ 
tematic approach to design using standards of quality to maximize learning 
outcomes (Pollacia, Russell, & Russell, 2009). Typically, online trainings 
provided are tailored to the individual online learning system vendors (Ke- 
arsley & Blomeyer, 2004) and less on course design itself (Rice & Dawley, 
2009). However, if effective course delivery begins with quality course de¬ 
sign (Wright, 2011), then how do we successfully teach educators the skills 
necessary to design a quality course? 


PURPOSE 

The purpose of this research was to determine whether a professional de¬ 
velopment model proposed by Mercer (2014) is a workable solution to teach¬ 
ing K-12 instructors to successfully apply essential standards of the Quality 
Matters Rubric to the design of a K-12 online course. Therefore, the present 
study sought to answer the following question: To what extent does participa¬ 
tion in an online professional development course centered on instructional 
design principles help K-12 educators design an online course that meets rec¬ 
ognized standards of quality? 


METHODS 

This study was conducted in the context of a professional development 
course delivered by the College of Education and Human Development at 
a large research university to K-12 teachers across the state over the course 
of approximately two years. The college is part of a subscribing institution 
to the Quality Matters program and supports the standards delineated in all 
three rubrics provided by the program. The Virtual Instructor Certificate Pro¬ 
gram began as a one-day workshop and has evolved over the course of sev¬ 
eral years into a fully online course based on fundamental design principles. 
VICP participants include working professionals, faculty in higher education, 
K-12 educators, and graduate students. 
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This specific study primarily focused on outcomes related to the essential 
standards of the QM Rubric fourth and fifth editions. Although QM provides 
a K-12 rubric, the timing of its creation along with the history of VICP using 
the higher education rubric with effective results perpetuated its continued 
use regardless of the participant’s professional role (e.g. K-12 educator). In 
addition, lead instructors for VICP had not yet been certified as K-12 QM 
peer reviewers and the differences between the two rubrics were not deemed 
substantial enough at the time this study took place to warrant changing the 
rubric used to assess final projects. However, VICP participants are now 
given the choice of which rubric they want used to assess their final project 
based on the type of course being designed. The following sections will de¬ 
scribe in further detail the participants, the VICP online professional devel¬ 
opment course experience, the instrument used to measure learning outcomes 
that was based on the QM Rubric, and data analysis procedures. 

Participants 

Participation in the Virtual Instructor Certificate Program is generally pro¬ 
vided at the school district level or based on individual interest. In this study, 
participants included both options. Participants paid individually or through 
their corresponding district. The sample for this study included K-12 teachers 
who completed the VICP course between 2014 and 2015. The total sample 
size was n=97 (female=70, male = 27). Participants were affiliated with eight 
school districts, one region service center, and one participant was a non-em¬ 
ploy ed K-12 teacher. 



Figure 1. Participants’ Subject Area Certifications. This figure illustrates 
the many areas within which participants were certified to teach. 
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Among the 97 submitted projects, 42 were scored using the Quality Mat¬ 
ters Higher Education Rubric (2011-2013) edition, and 49 were scored us¬ 
ing the Quality Matters Higher Education Rubric 5th edition. Moreover, 
participants had a variety of subject areas certified by the state documented 
certificates, with many teachers holding multiple certifications. Figure 1 
shows the subject areas within which the teachers were certified to teach. 

Professional Development: The Virtual Instructor Certificate Program 

The Virtual Instructor Certificate Program is a state-approved profes¬ 
sional development provider through the state’s virtual school network. 
However, the course is available to anyone interested in learning founda¬ 
tional concepts of instructional design within the context of digital learn¬ 
ing. Participants typically include faculty, graduate students, persons work¬ 
ing in public education (the focus of this study), as well as educators work¬ 
ing in non-traditional environments. VICP is a six-week course delivered 
completely online. The course prepares participants to design, develop, and 
manage an online course with specific emphasis on hands-on activities and 
feedback centered on standards-based design. Course content is founded on 
fundamental instructional design principles and best practices for deliver¬ 
ing online courses. Topics include but are not limited to learner and context 
analysis, writing measurable learning objectives, designing course activi¬ 
ties that align with objectives, creating effective instructional strategies, and 
preventing academic dishonesty. Upon completion of the program, partici¬ 
pants have a fully developed online module embedded in the context of a 
fully online course that is consistent with essential standards of the Qual¬ 
ity Matters Higher Education Rubric. This product provides the participant 
with a working template for completing the remaining modules within their 
online course. Participants are awarded a certificate for 60 hours of profes¬ 
sional development. If the participant holds a current state teacher certifica¬ 
tion, they are then able to teach online courses in the state’s approved vir¬ 
tual network. 

Instrumentation 

VICP supports participants as they design and develop an online module 
within the context of a fully online course. This online module is evaluated 
at the end of the six-week course using the essential standards of the Quality 
Matters Higher Education Rubric. The rubric is comprised of eight general 
standards. Each general standard is then broken down into varying numbers 
of specific review standards. Standards deemed “essential” are worth three 
points and are required to be met in order for an online course to potentially 
be what is referred to as a “Quality Matters recognized course” (See http:// 
www.qmprogram.org for detailed rubric information). 
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During the time frame of this investigation. Quality Matters published a 
new edition of its Higher Education Rubric based on new research. Thus, de¬ 
pending on the time a participant completed VICP, two versions of the Qual¬ 
ity Matters Higher Education Rubric were used to evaluate projects - the 
4th edition (2011 - 2013) and the 5th edition. Differences between the two 
versions of the rubric were minimal upon comparison with regards to essen¬ 
tial standards. For example, Standard 6.3, “Navigation throughout the online 
components of the course is logical, consistent, and efficient” from the 4th 
edition is now included in Standard 8.1 of the 5th edition, “8.1 Course navi¬ 
gation facilitates ease of use.” These minor numbering changes are noted in 
the data analysis section and labeled for ease of use. 

Both versions of the rubric contain the same 21 essential standards, one 
of which (8.1 in the 4th edition; 8.2 in the 5th edition) is not taught during 
the course of VICP to the extent to which assessment would be valid. Thus, 
this standard is marked as “met” with a note telling participants that it is not 
evaluated and giving recommendations to pursue further study of its require¬ 
ments. This standard has to do with creating an accessible course that is 
deemed out of the scope of the six-week program. 

Although participants in the program, and for this study, were from the 
K-12 environment, the program also includes higher education faculty. Be¬ 
ginning in 2014, the program began to see an influx of participants from pub¬ 
lic school educators in K-12. However, instructors for the program were cer¬ 
tified Quality Matters Peer Reviewers and experts on the Higher Education 
Rubric. When comparing the K-12 Rubric with the Higher Education Rubric, 
minimal discrepancies were noted, especially with regards to essential stan¬ 
dards. Therefore, use of the Higher Education Rubric to evaluate course out¬ 
comes for all participants was deemed appropriate during the course of this 
study. VICP K-12 participants are now given the option to have final projects 
evaluated using the K-12 Rubric. 

Lead instructors for the program are responsible for mentoring each par¬ 
ticipant as they design an online module within the context of a full course 
in order to meet the essential standards. These instructors are trained using 
a four-phase process. First, they complete the VICP course as a participant. 
Second, they are required to shadow an experienced lead instructor to learn 
how the course works from the instructor perspective. During this time they 
meet weekly with the program director to discuss questions and figure out 
how to interact with participants and guide them to success. Third, they co¬ 
teach with the director to gain experience grading and providing appropriate 
feedback. Finally, they are given full responsibilities with monitoring of the 
director to help with any possible issues of understanding how to determine 
whether the course does or does not meet an essential standard and why. All 
VICP lead instructors are experienced instructional designers and extensively 
trained in Quality Matters. This process increases the likelihood that a partic¬ 
ipant will receive the same feedback regardless of who is teaching the course. 
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Data Sources and Analysis 

The current study used an existing data set collected by the college from 
previous VICP cohorts. Unique identifiers were not included in the data set. 
Data was collected from K-12 teachers who completed VICP between Janu¬ 
ary 2014 and December 2015. Scoring procedures model that of QM’s all or 
nothing scoring method. A module meeting the essential standard was award¬ 
ed three points. If the standard was not yet met, a score of 0 was awarded 
with feedback provided and a request for a resubmit. Participants worked 
with a VICP instructor to revise the module until all essential standards were 
met. 

A quantitative research method was used and descriptive statistics were 
computed to answer the research question. Two analyses were conducted. 
First, counts and frequencies were computed using an Excel spreadsheet for 
all participants assessed using either version of the Higher Education Rubric. 
To determine the extent to which VICP was effective in helping participants 
to apply standards of quality to an online course, the number of teachers who 
met each essential standard in their initial submission were counted. The 
counts were used to compute the frequency of the participants who met the 
essential standards assessed. It is important to note that these K-12 teachers 
did not have existing developed courses. Without an existing course, it was 
not possible to gather any preliminary data prior to the completion of VICP. 

Second, some participants were affiliated with school districts with set 
online infrastructure in place prior to their teachers taking the course. Thus, 
the data was further analyzed to explore whether the existence of a distance 
education infrastructure (i.e. learning management system, shared technical 
support, and accessibility information page) in place affected outcomes. Two 
subgroups were created (i.e., districts with infrastructure and districts with¬ 
out infrastructure). The number of teachers who met each essential standard 
was counted, and the frequency of the projects that met the essential stan¬ 
dards was computed within each subgroup. The data from the subgroup was 
analyzed to compare districts with and without infrastructure. The following 
section will discuss the results based on individual standards. The reader is 
encouraged to visit the QM website to view specific standards as they are 
discussed in relation to research outcomes (www.qualitymatters.org/rubric). 


RESULTS 

A large percentage of program participants met many of the essential stan¬ 
dards in their initial submission (See Figure 2). In particular, 95% (n = 86) 
met standard 1.2, 98% (n=89) met standard 2.5, 98% (n=89) met standard 
6.2, and 99% (n=90) met standard 7.1. Participants performed above aver¬ 
age on standards 2.1, 3.2, 5.1, 7.2, and 6.3/8.1, meeting these standards at 
a frequency between 84% and 87%. Participants performed somewhat 
poorly on standards 2.4, 3.3, and 5.3, respectively, 61% (n=55) met stan¬ 
dard 2.4, 63% (n=57) met standard 3.3, and 59% (n=54) met standard 6.2. 
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Figure 3 illustrates the count of teachers who met each essential standard 
based on rubric edition. 


Frequency of Participants Meeting Standards Evaluated by 
Higher Ed Rubrics 



8.1 

Figure 2. Frequency of participants meeting standards. 


This figure illustrates the percentage of participants who met each essential 
standard from both the 4th and 5th edition of the Higher Education Rubric. 
Participants who were assessed based on the K-12 rubric were not included. 

The Number of Participants Meeting Each Standard Evaulated by 
Higher Ed Rubrics 
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Figure 3. Number of participants meeting each essential standard. 
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Figure 3 shows the specific count of participants who met the standard 
based on the rubric used to assess the final project. 

Infrastructure vs. Non-infrastructure 

Results indicated that approximately half of the participants from school 
districts with infrastructure did not generally meet standards 2.4, 3.3, and 
5.3, with only 54% (n=59) meeting standard 2.4, 56 % (n=59) meeting stan¬ 
dard 3.3, and 52% (n=60) meeting 5.3. Figure 4 displays how often partici¬ 
pants who had some infrastructure in place met each of the essential stan¬ 
dards of the 4th and 5th edition of the QM Fligher Education Rubric. 


Frequency of Participants Meeting Standard from a School 
District with Infrastracture 



8.1 

Figure 4. Frequency of participants meeting standard from a school district 
with infrastructure. 

This figure illustrates the extent to which participants met essential stan¬ 
dards who were also affiliated with a district that had an online learning in¬ 
frastructure in place. 

While participants without a set infrastructure (n=31), were relatively 
successful in meeting standards for either the 4th or 5th edition of the Qual¬ 
ity Matters Fligher Education Rubric, only half of these participants met 
standard 7.2. Figure 5 depicts essential standards where infrastructure may 
play an important role. 
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Frequency of Participants Meeting Standards from a School 
District Without Infrastructure 



8.1 


Figure 5. Frequency of participants meeting standards from a school district 
without infrastructure. 


DISCUSSION 

Teachers in K-12 have long carried out the duties of an instructional de¬ 
signer without having a formal name for the process to which they commit 
much daily work (Gyabak, Ottenbreit-Leftwich, & Ray, 2015). However, de¬ 
signing an online course requires a distinct set of skills (Wolcott & Shattuck, 
2007). One important skill is that of learning to design a course such that 
it meets recognized standards of quality in order to maximize learning out¬ 
comes (Budzick, 2014; Mercer, 2014). 

The present study explored the extent to which a six-week, authentic, 
project- and standards-based online professional development course helped 
K-12 teachers design a quality online course. Similar to Wayer (2013), out¬ 
comes suggest that the course supported transference of concepts to the de¬ 
sign of an online course. Results indicate that 84% of participants met 15 out 
of the 20 essential standards assessed in their initial project submission. Out 
of these 15, areas of success appear to be within the areas of course introduc¬ 
tion (General Standard 1), learning objectives (General Standard 2), course 
technologies (General Standard 6), learner support (General Standard 7), and 
usability (General Standard 8). In fact, at least 90% of participants met 10 
of the essential standards, with almost half of those standards centered on 
learning objectives. Learning objectives are foundational to the field of in¬ 
structional design and central to alignment, a concept espoused by Quality 
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Matters to be pivotal to online course quality (qmprogram.org). In fact, on 
average, 90% of participants met standards related to alignment (with the ex¬ 
ception of 3.1 noted in the next paragraph) upon initial submission. 

Similar to findings of Mercer (2014), the area of greatest concern was in 
assessment (General Standard 3). Specifically, issues arose in standards 3.1 
and 3.3 that focus on aligning assessment with learning objectives and pro¬ 
viding appropriate explanation for how the assessment will be evaluated. 
Seventy-eight percent of participants met Standard 3.1, and only 63% met 
Standard 3.3. This raised a red flag given the importance of aligning assess¬ 
ment with objectives and providing adequate explanations for students to un¬ 
derstand how to meet instructor expectations for those assessments. 

Other areas for concern included one topic related to learning objectives 
(Standard 2.4) and another focused on instructional materials (Standard 4.2). 
Sixty-one percent of participants did not adequately demonstrate the relation¬ 
ship between the course activities and the learning objectives, and only 76% 
of participants clearly explained to students the purpose of the instructional 
materials. 

The standard with the lowest percentage of participants was Standard 5.3. 
This standard is about response time on the part of the instructor. Only 59% 
of participants met this standard upon initial submission, despite the fact that 
within one assignment, participants are given a checklist and are asked to 
provide this information within the course as one of the items on that list. 
Although this was the standard met least among all participants, overall stan¬ 
dards related to this area were met by at least 84% of the participants, and, 
thus, General Standard 5 was not noted as a primary area of concern. 

Roughly half of the participants were assessed on each version of the 
Higher Education Rubric. Thus, outcomes of comparing counts of par¬ 
ticipants who were assessed by the two editions indicate that two standards 
show a decline in success. The annotation for Standard 2.4 changed from the 
4th to the 5th edition. Therefore, the ways QM suggests meeting this stan¬ 
dard changed. This could account for the decline on this particular standard. 
However, participants performed better when assessed with the 4th edition 
on Standard 5.3 yet this annotation did not change. Further investigation is 
needed to determine possible reasons for this outcome. 

Participants from school districts without infrastructure outperformed 
participants from school districts with infrastructure on almost all essential 
standards except Standard 7.2. According to Marek (2009), infrastructure 
has been shown to positively impact quality. Therefore, the result may seem 
unexpected. However, most standards are dependent on the work of the de¬ 
signer and not the infrastructure within which the course is housed, with the 
exception of 7.2. More than 90% of participants from school districts with in¬ 
frastructure met standard 7.2, as opposed to 55% of participants from school 
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districts without infrastructure. Standard 7.2 is about institutional policies re¬ 
lated to accessibility. If the administration has created an infrastructure that 
values accessibility, then most likely this standard is easy to meet and only 
requires awareness on the part of the course designer. Interestingly, these 
same participants performed poorly on standards 2.4, 3.3, and 5.3, similar to 
what resulted in the full sample aggregate. However, they outperformed par¬ 
ticipants from school districts with infrastructure on these standards, suggest¬ 
ing that prior administrative infrastructure had little impact on participants’ 
performance and characteristics of the K-12 educator played a larger role in 
course outcomes. 


LIMITATIONS 

The results of this study should be considered within the context of vari¬ 
ous limitations. As mentioned earlier, in order to conduct the number of 
cohorts of the VICP course, it is essential to use more than one instructor 
throughout the year. These instructors are trained using protocol for how 
to interact with participants, types and extent to which feedback should be 
provided, as well as how to determine if a standard has been met. However, 
there is always a level of subjectivity to determining whether a standard has 
been met. In addition, some instructors may provide more extensive feed¬ 
back than others for reasons such as experience, time available, or interest 
in the project. Furthermore, each participant in the VICP course comes from 
a background of knowledge that was not included in this study. Some par¬ 
ticipants have experience teaching online, while others do not. These back¬ 
ground characteristics are sure to influence the degree to which one partici¬ 
pant meets a standard when another does not, even though both may work 
for the same district and have access to the same resources. These character¬ 
istics could influence course outcomes as opposed to the course experience 
and feedback provided. Finally, no pre-test data was collected because par¬ 
ticipants did not already have fully developed courses that could be assessed 
according to QM standards. 


CONCLUSION 

The present study explored outcomes of a professional development 
course focused on supporting K-12 educators to create an online course that 
meets national standards of quality. Current research and national guidelines 
recommend that this type of professional development should be authen¬ 
tic, project-based, and standards-based. Outcomes demonstrate this model 
for professional development can successfully prepare participants to de¬ 
sign an online course to meet standards. The VICP provided participants an 
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authentic learning experience by which they took the role of online student 
for six weeks. The course required that participants complete a project con¬ 
sisting of a fully online module of instruction within the context of a fully 
online course. This project as assessed using the Quality Matters Higher Edu¬ 
cation Rubric, a set of nationally recognized standards for assessing the qual¬ 
ity of an online course. 

To determine the extent to which K-12 educators were able to apply na¬ 
tional standards of quality to an online course, data was collected from initial 
project submissions to analyze the number and type of standards met after 
completion of the VICP course. A majority of the participants were able to 
meet 15 of the 20 essential standards used to determine quality of the mod¬ 
ule. All but one standard related to alignment were met by almost all par¬ 
ticipants demonstrating this model of professional development is successful 
in helping participants understand and apply the concept of alignment to an 
online course. 

In comparison to prior research on this topic, standards related to assess¬ 
ment are still of great concern and need further exploration on how best to 
teach course designers how to design assessments to meet quality standards. 
In addition, participants need further support in understanding how to dem¬ 
onstrate the relationship between course activities and learning objectives as 
well as sharing the purpose of the learning materials and expected response 
time as it pertains to course feedback. 

Participants who were affiliated with a district that did not have an infra¬ 
structure for online learning in place outperformed participants who were 
affiliated with districts who did. Most likely this is because most essential 
standards are specific to the teacher’s preferences and belief system with re¬ 
gards to each standard’s requirements. The primary example from this study 
was Standard 5.3 that requires instructors to provide a time frame for re¬ 
sponding to students. Districts without a set infrastructure would have no 
such mandated time frame, and thus meeting the standard is left to the discre¬ 
tion of the instructor and their support of this practice. On the other hand, 
Standard 7.2 addresses the importance of including accessibility policies. 
Districts with infrastructure in place can provide an accessibility home page, 
which may be why participants from school districts with infrastructure typi¬ 
cally met this standard. Although infrastructure is necessary for learner suc¬ 
cess (Marek, 2009), there are sure to be other factors to consider. 

This study was significant in providing empirical evidence of the effects 
of a specific professional development model purposed with preparing edu¬ 
cators to design online courses to meet national standards of quality. As noted 
in the literature (Cavanaugh, Barbour, & Clark, 2009; Mercer, 2014; Shat- 
tuck, 2013), there is very little research providing guidelines on how best 
to teach this distinct set of skills (Wolcott & Shattuck, 2007). Although the 
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outcomes were positive, there were concerns that arose with regards to spe¬ 
cific standards that participants did not meet for reasons that are not clear and 
need further investigation, in addition, limitations to the study such as mul¬ 
tiple instructors across cohorts, varying amounts of feedback, and unknown 
learner characteristics necessitate that a more controlled study be conducted. 

Future research should include pre and post data to provide stronger ev¬ 
idence as to the effectiveness of the program. For example, Mercer (2014) 
conducted a t-test using pre and post-test data regarding faculty knowledge 
of best practice in online course design. A similar instrument could be used 
to measure participants’ prior understanding of best practice in online course 
design and used to determine the extent to which VICP improves this knowl¬ 
edge. In addition, inter-rater reliability among lead instructors could further 
be demonstrated using a sample of participant submissions whereby com¬ 
parisons can be made regarding assessment outcomes across instructors. Fi¬ 
nally, future research should include more information regarding participant 
characteristics and investigate course outcomes across instructors according 
to instructor characteristics so as to provide further clarity on outcomes that 
can more confidently be attributed to the VICP course. 
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